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We study the classical and quantum phase transitions ol Sp(4) spin systems on three dimensional 
stacked square and triangular lattices. We present general Ginzburg-Landau field theories for various 
types ol Sp(4) spin orders with different ground state manifolds such as CP(3), S"^ /Z2, Grassmann 
manifold 62,5, G2,& and so on, based on which the nature of the classical phase transitions are 
studied, and a global phase diagram is presented. The classical phase transitions close to quantum 
phase transitions toward spin liquid states are also discussed based on renormalization group (RG) 
flow. Our results can be directly applied to the simplest Sp(4) and SU(4) Heisenberg models which 
can be realized using spin-3/2 atoms and Alkaline earth atoms trapped in optical lattice. 



1, INTRODUCTION 

For decades condensed matter physicists have been ac- 
tively studying the spin systems with large symmetries 
such as SU(N) and Sp(N) i, S i, % i, 0, i, i, US], 
mainly motivated by the fact that under large-N general- 
ization the semiclassical spin order with spin symmetry 
breaking is weakened, and even vanishes completely be- 
yond certain critical Nc. A good example is the SU(N) 
Heisenberg model on square lattice with fundamental 
and conjugate fundamental representation on two sub- 
lattices (FCF Heisenberg model), which for N > 4 is 
quantum disordered, and for N < 4 the ground state 
spontaneously breaks the SU(N) symmetr y, w ith ground 
state manifold (GSM) CP(N- 1) [1, Very re- 

cently it was proposed that, without fine-tuning any pa- 
rameter, the SU(N) spin systems with N as large as 10 
can be realized by alkaline earth atoms trapped in opti- 
cal lattice [l3|, so the large-N spin system is no longer 
merely theoretical toy. Many previous works showed that 
for the special value N = 4, the Sp(4) symmetry can be 
realized with spin-3/2 fermionic atoms, and when the 
spin-0 and spin-2 s— wave scattering lengths are e qual , 
the system has an even larger SU(4) symmetry [IJ]. 
Motivated by these observations, quantum magnetism 
based on the spin-3/2 atoms has been actively studied 

Although the GSM of the ordered state of SU(N) FCF 
Heisenberg model on square lattice has been identified as 
CP(N— 1) long ago, a detailed Ginzburg-Landau (GL) 
field theory for this ordered state has not been thoroughly 
studied. A GL theory of this state can answer the fol- 
lowing question: Suppose the SU(N) Heisenberg model 
is defined on the 3d cubic lattice with CP(N - 1) GSM, 
what is the finite temperature transition between this or- 
dered phase at low temperature and a disordered phase 
at high temperature? For N = 2, this question is fairly 
simple, because CP(1) — S"^, the finite temperature tran- 
sition is no more than one single 3d 0(3) transition. For 
larger-A^ cases, the question is complicated by the fact 
that CP(N — 1) manifold does not have a general simple 
parametrization as CP(1). The standard way to param- 



eterize the CP(N — 1) manifold is to treat it as N com- 
ponent of complex boson coupled with U(l) gauge field, 
while keeping the SU(N) global symmetry of the action, 
but this parametrization of CP(N — 1) manifold fails to 
describe the finite temperature phase transition, which 
will be discussed in the next section. Therefore we need 
to write down a GL theory based only on the physical 
observable order parameters. 

In the current work we will focus on the case with N = 
4 and discuss the finite temperature phase transition of 
system with CP (3) GSM. One sample system which has 
CP(3) GSM is the Sp(4) Heisenberg model on bipartite 
lattice with one particle per site: 



<ij> 



(1) 



with a = 1...5 are five 4x4 Gamma matrices, and 
pab ^ l-\r'\T^] are 10 generators of Sp(4) ~ S0(5) 
group. Here we choose the following standard conven- 
tion of Gamma matrices: 



-pa 



1,2,3, 



1, = /i^ ® 1.(2) 



For arbitrary Ji and J2 this system has Sp(4) symme- 
try, while when Ji = J2 this model is equivalent to the 
SU(4) FCF Heisenberg model [i3|. Ji and J2 can be 
tuned with spin-0 and spin-2 s— wave scattering lengths 
of spin-3/2 cold atoms 17|. Our formalism suggests that 
for a Sp(4) spin system on the 3d cubic lattice with GSM 
CP(3), depending on the ratio J1/J2 the classical phase 
diagram can have different scenarios. The most interest- 
ing scenario is the region J2 > Ji in model Eq. [l] at 
finite temperature there are two transitions, with one 3d 
0(5) transition followed by a 3d 0(3) transition at lower 
temperature. On stacked triangular lattice, it was shown 
that the Sp(4) Heisenberg model Eq. [l]has \/3 x %/3 spin 
order with GSM S'^/^2 [2l|. At finite temperature again 
there can be two transitions, with one 3d 0(5) transi- 
tion followed by a "coupled" 0(3) transition. Besides 
CP(3) and S"^ /Z2, many other spin symmetry breaking 
semiclassical states of Sp(4) spins with different GSM can 
exist, especially for half- filled (2-particle per site) system, 
which will also be discussed in this work. 
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This paper is organized as follows: In section II, we will 
study the GL theory of the Neel and -s/S x V3 phases of 
the Sp(4) spin system on three dimensional lattices, and a 
global phase diagram is presented. Sp(4) spin states with 
other GSM such as Grassmann manifold 6*2,5, G2,e and 
SO(5)/SO(3) will also be discussed, with applications to 
half-filled Sp(4) spin models. Our GL theory can also be 
used to distinguish different GSMs with the same dimen- 
sion and similar quotient space representation. In section 
III we will study the classical phase transitions close to 
quantum phase transitions between ordered and spin liq- 
uid phases. In section IV we will briefly discuss a more 
exotic manifold, the "squashed S"^" and its potential to 
be realized in Sp(4) spin systems. 



2, GL THEORIES FOR SP(4) SPIN SYSTEMS 
A, Collinear Phases 



the action ([3|) should be supplemented with the "feature- 
less" monopole, which is relevant at least for small N 
at the critical point r = 0. For N = 2, the "trivial" 
monopole drives the CP(1) model to the 0(3) universal- 
ity class, but for larger N there is no such simple relation. 
Therefore the CP(N — 1) model plus monopole does not 
tell us much about the nature of the transition in gen- 
eral, and we need another convenient way to describe the 
CP(N- 1) manifold. 

Therefore, to describe the GSM and transition we need 
to introduce a linear sigma model at 4 — e dimension 
with gauge invariant order parameters, in the form of 
zj^z/^. There are in total 15 independent bilinears of this 
form, which can be simply rewritten as the following 5 
component vector and 10 component adjoint vector: 



)ab 



Z^TabZ, 



(5) 



Let us now consider the Sp(4) Heisenberg model Eq. 
[T] on the 3d cubic lattice with 1 particle per site. On 
the 2d square lattice, both analytical and numerical re- 
sults conclude that at the special point Ji — J2 with en- 
larged SU(4) symmetry, the ground state of this model 
has semiclassical order [1, Ell, 113, with GSM CP (3), 
which extends into a finite range of the phase diagram 
tuned by J2 / Ji [2l| . The semiclassical order is expected 
to be stable with the third direction unfrustrated inter- 
layer coupling. In this Neel phase, both F"'' and F" are 
ordered. For instance, we can take the trial single site 
state \tp) = (1,0,0,0)*, and it is trivial to see that it has 
nonzero F^, F^s and F^^. 

As already mentioned in the introduction, the standard 
way to parameterize the CP(N — 1) manifold is to treat 
it as N component of complex boson coupled with U(l) 
gauge field, while keeping the global spin symmetry of 
the action: 



2\2 
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This action is written down based on the fact that 



CP(N- 1) = S^^-^/\J{l). 



(3) 



(4) 



Here 5^^^^ represents the GSM of the condensate of N 
component of complex boson, and U(l) represents the 
U(l) gauge field A^. However, at finite temperature, a 
simple CP(N — 1) model in Eq. [3] on three spatial di- 
mension would lead to a wrong transition, because this 
model describes a transition between the ordered phase 
and a photon phase. However, finite temperature in- 
duces finite density of monopoles of A^, which will change 
the photon propagator at long scale. The disordered 
phase is generically identical to the "confined phase" with 
monopole proliferation and no lattice symmetry break- 
ing i.e. the monopoles without Berry phase. Therefore 



The complex bosonic field Za are the low en ergy 



Schwinger bosons of Sp(4) spin system. In Ref. [2l|, 
it was shown that in the Neel order cf)"-'' is the staggered 
order (— 1)T°^ while the 0(5) vector is the uniform 
order F°, which can be naturally expected from Eq. [U 
when Ji and J2 are both positive. However, 0"'' and (j>'^ 
are not independent vectors, because the Sp(4) symme- 
try of the system allows for coupling between these two 
vectors in the free energy, which can be manifested by 
the following identities: 
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a 

Tvra ibc ide 

N = eabcde<P <P ■ 



(6) 



^abcde IS the fivc dimcnsional antisymmetric tensor. Also, 
the five F° matrices are all constructed by bilinears of the 
spin-3/2 operators, while F"'' are constructed by linear 
and cubics of the spin operators [li, 17|. Therefore $ is 
time reversal even, and identical to the nematic 0(5) vec- 
tor iV" = eabcde(t>^''(t>'^'' in the ordered state of the CP (3) 
model with \z\'^ = 1. 

Now we can write down a classical GL theory for Sp(4) 
spin system with CP (3) GSM: 



ab.fi 

I a ibc ide 
l^abcde<P 9 
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YY 



ah 



(7) 



The ellipses include all the other terms allowed by Sp(4) 
global symmetry. When ri = r2, this free energy is 
SO(6)'^SU(4) invariant, which corresponds to the point 
Ji = J2, where the model is equivalent to the SU(4) FCF 
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Heisenberg model on the cubic lattice. We can also view 
the adjoint vector 0"'' as an 0(10) vector which originally 
should form a GSM S^, and the cubic term 7 makes the 
ten component vector 0°^ align in a 6 dimensional sub- 
manifold of where the 0(5) vector ^ £abcde(t>^'^4>'^^ 
is maximized. 

A global mean field phase diagram can be plotted 
against r = ri + r2 and Ar = ri — r2, as shown in Fig. 
[TJ The parameter r is tuned by temperature, and Ar is 
tuned by AJ = Ji — J2, which is evident with the ob- 
servation that Ar = corresponds to the same SU(4) 
point AJ ~ and both finite Ar and AJ violate the 
SU(4) symmetry. There are three different regions in 
the phase diagram. Close to the SU(4) point Ar — 0, 
the cubic term 7 drives a first order transition at the 
mean field level, with both {(j)"'^) and {(j)"") jump discon- 
tinuously. The first order transition extends to a finite 
region in the phase diagram. The second region of the 
phase diagram has Ar < (Ji > J2), here t/)"^ wants to 
order before 0", but due to the 7 term in the free energy 
(O, the order of 0"'' implies order of Therefore in 
this region the phase transition can be safely described 
by a free energy in terms of only 0°^, after integrating 
out 

ab,fj. a 

+ 92{j2i'i^'"'rr+--- (8) 

ab 

Here 72 < to make sure the ground state wants to 
maximize 0°. We can treat 72 as a perturbation at the 
3d 0(10) transition, and a coupled renormalization group 
fiow of 72 and 172 will determine the fate of the transition. 

The third region is Ar > ( Ji < J2), now 0° tends to 
order before and there are in general two separate 
second order transitions at finite temperature, with 0" 
orders first. The transition of 4'°' is a three dimensional 
0(5) transition. After the ordering of 0°, the symmetry 
of the system breaks down to 0(4). Let us take the 
expectation value of cj) as (0) = (cr, 0, 0, 0, 0), the coupfing 
between 0" and in free energy ([7]) reads: 

ea.c.e^"/^^'^^ = Tic^^'d^"' - cf"' + ^'^0^^). (9) 

Now one can diagonalize the quadratic part of the Eq. [7] 
and Eq. [9l the eigenmodes are characterized by the rep- 
resentation of the residual 0(4) ~ SU(2) x SU(2) sym- 
metry. The residual 0(4) symmetry group is generated 
by 6 matrices F"'' with a, 6 ^ 1. The two SU(2) normal 
subgroups of 0(4) are generated by matrices — F^'^ + r''^, 
p24 _,_ p35^ _p25 _^ p34 (denoted as subalgebra su(2)^) 
and + F-^s^ -F24 + F35, F^s + T^^ (denoted as sub- 
algebra su(2)^) respectively. We will decompose the 10 
component vector based on the representation of the 
su(2)^ and su(2)^ algebras, different representations will 
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FIG. 1: The phase diagram of GL theory Eq. [T] plotted 
against r = ri + r2 and Ar = ri — r2. The red line is a 
first order transition, the blue line is a 3d 0(5) transition, the 
green line is a 3d 0(3) transition. The golden line is a second 
order transition at the mean field level, the true nature of the 
transition can be obtained by a detailed RG calculation for 
Eq. |S]with 72 < 0. A similar phase diagram can be applied 
to Eq. [14] for the stacked triangular lattice, with the green 
line representing a coupled 0(3) transition described by Eq. 

m 



have different eigenvalues: 

g'(z = i...4) = (</>i2, </,i3, </.",0i^), 

eigenvalue : r, Representation : 0(4) vector; 
fA(. = l,2,3) = (-023-f</.^^ cj^'' + cb'\ -cb^' + cb^'), 
eigenvalue : r — 7cr, Representation : (1, 0); 

n{^ = 1,2,3) = (<^23+0^^ -(/.2^+<^=^^ cb^'+cb^'), 

eigenvalue : r + 70-, Representation : (0, 1). (10) 

Here Ta and Tb transform as vectors of SU(2)^ and 
SU(2)^ respectively. Notice that although SU(2)^ and 
SU(2) ^ are both normal subgroups of the SO (4) after 
the order of (j)"" , neither of them can be normal subgroup 
of the original S0(5) group, because S0(5) group is a 
simple group while SO (4) is a semisimple group. 

If 7(7 > 0, Ta has the lowest eigenvalue, so the 0(3) 
vector Ta will order after 0" . The main question is which 
universality this transition belongs to. Since Q and Tb 
are massive and only have short range correlation at the 
transition of Ta, integrating out them will not induce 
any critical behavior for Ta, and hence the Goldstone 
mode of 0° after its ordering is the biggest concern. The 
Goldstone mode (0, tti, 7r2, tts, 7r4) forms an 0(4) vector, 
and the Goldstone theorem guarantees its gaplessness. 
The simplest coupling one can write down with these 
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constraints is: 



CP(3) 



2,5 



(11) 



Y,<0 



y,= o y,>o 



This term only generates irrelevant perturbations at the 
0(3) transition of Ta after integrating out tt. Notice 
that couplings like {Ta)^{tt)'^ though preserves the global 
0(4) symmetry, violates the Goldstone theorem after in- 
tegrating out Ta, as a mass gap ^ (T|) is induced for 
TT. Therefore now we can safely conclude that the phase 
transition of T4 is a 3d 0(3) transition. Notice that 
vector Tb and Q no longer have to order at lower tem- 
perature, because of the repulsion from ordered Ta, due 
to the quartic terms in Eq. [71 

After the ordering of Ta, the symmetry of the sys- 
tem is broken down to SO (2) x SO (3). The first S0(2) 
corresponds to the residual symmetry of SU(2)^ af- 
ter the order of Ta, and the second SO (3) corre- 
sponds to the SU(2)^ associated with Tb, therefore 
CP (3) manifold can also be written as quotient space 
SO (5)/ [SO (2) X SO (3)]. However, we should be care- 
ful about this formula, because there are two different 
types of so(3) or su(2) subalgebras of so(5). Besides 
the subalgebras su(2)^ and su(2)^ we used earlier, there 
is another SU(2) subgroup which is the diagonal sub- 
group of SU(2)^ X SU(2)^, we denote this subgroup as 
SU(2)y, which is no longer a normal subgroup of 0(4). 
The elements in algebra su(2)y are the linear combina- 
tion of the corresponding elements in su(2)^ and su(2)^: 

For instance, in the half-filled (2 particles per site) 
spin-3/2 cold atoms, one can naturally obtain an ordered 
state with ((--1)T'^'') ^ but with no order of 
which means that for this case action Eq. [5] is still appli- 
cable, while the sign of 72 is positive i.e. it corresponds 
to a different anisotropy of the manifold formed by 
the adjoint vector which minimizes the vector ^ 



^abcde 



Lbcxde 



(in contrast to CP(3)) (Fig. [21). In this case 
the GSM can still be written as SO(5)/[SO(2) x S0(3)], 
but here S0(3) is SU(2)y. For instance if {{-IfT^'^) ^ 0, 
the SU(2)^^ is generated by F^^, F^s and F^s. This GSM 
SO(5)/[SO(2) X S0(3)] with S0(3) - SU(2)y is called 
Grassmann manifold 6*2,5, which is mathematically de- 
fined as the set of 2-dimensional planes in 5 dimensional 
vector space [22j . 

The mean field phase diagram for the half-filled Sp(4) 
system tuned by the spin-0 and spin- 2 s— wave scatter- 

I . Besides 
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ing lengths is studied in Ref. 
the phase with ((— 1)T°^) ^ discussed in the pre- 
vious paragraph, there is another typical phase with 
((-l)T'^) ^ and GSM SO(5)/SO(4) = S^. These two 
phases are separated from each other by the SU(4) point 
with equal spin-0 and spin-2 scattering lengths, where 
due to the enlarged symmetry, the two different orders 



should have equal energy Suppose ((— 1)T^^) is 
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FIG. 2: The schematic ground state manifold phase diagram 
of (Fig. a) Eq. [Hand (Fig. b) Eq. [lH with Ar = n - r2. 

try of this order is generated by F^^ F^^, F-*^, F'^^ F^, 
F^ and F^ which form subgroup SO (2) x SO (4) of the 
S0(6) ^ SU(4) symmetry group. More detailed analysis 
would show that now the GSM is the Grassmann man- 
ifold SO(6)/[SO(2) x S0(4)] = G2,6 (Fig. ^, which is 
defined as the set of 2 dimensional planes in 6 dimen- 
sional vector space. 

One can write down a GL field theory for the half-filled 
Sp(4) spin system as follows: 

Fhf = 5](V^(/)'^^)2-f (V^0'^)2-Hri(0'^'')2-f r2(0'')2 

a a 

(12) 



ab 



nonzero at this SU(4) point, now the residual symme- 



ri = r2 corresponds to the SU(4) point, and r2 < ri 
(r2 > ri) corresponds to the case with ((— 1)'F") ^ 
(((-l)T"'') ^ 0). Notice that the cubic term 
l^abcde4'°'4>^'^<j}'^'^ IS uot allowed here because 0° and <j)°'^ 
both represent staggered orders, so this cubic term would 
switch sign under lattice translation. The ellipses in 
Eq. [T2I includes other terms allowed by symmetry, for 

instance Y.ab{'^abcde<j)''(j)'^''Y ■ 

In 2+1 dimensional space, another possible ground 
state around the SU(4) point of the half-filled system is 
the algebraic spin liquid, which has been actively studied 
analytically (tLN, |23|, |2j, |25|, 126| and has gained numer- 
ical supports [27|. However, the fate of the SU(4) point 
at three dimension is unclear, so in this work we tenta- 
tively assume it still has magnetic order which bridges 
the orders on two sides of the phase diagram in Fig. [2f>, 
and the transition between the two different spin order 
patterns at zero temperature should be first order. 



B, Noncollinear Phases 

Now let us move on to the GL theory for Sp(4) s pin 
system with noncollinear spin orders. It was shown |21| 
that the GSM of the ordered phase of Sp(4) system on 
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the triangular lattice is S'^/Z^ with \/3 x ^/3 order of T"^ 
and coUinear and uniform order of F". By tuning Jij J\ 
there is a transition between the ordered phase and a de- 
confined Z2 spin liquid which belongs to the 3d 0(8) uni- 
versality class. Now let us consider the Sp(4) Heisenberg 
model on the stacked triangular lattice, and study the 
GL theory in terms of physical order parameters. This 
ordered state is characterized by the a/S x \/3 order of 
0f -I- i<^f = z*F"''z, and a uniform order of (^^ = z'^Vz. 
Za is the Sp(4) bosonic spinon expanded at the minima 
of the spinon band structure, which are located at the 
corners of the hexagonal Brillouin zone Q = (±47r/3, 0). 
The two 10 component Sp(4) adjoint vectors cfil'^ and (/12'' 



are "perpendicular" to each other: 



the ordered state. The vectors 
following relations: 



6f, 



,</)?Vf = 0. In 
and (j)°- satisfy the 



ihc ide 
^abcdeVl <Pl 



^abcde<l>2 <t>\ 



\z\^c^^ 



(13) 



Therefore the GL theory reads: 
2 



F = ^^(V,0f)2+ri(CY + (V,0'^)' + ^2(0")' 

2—1 a, 6 



ab 



ab 



cd 



The last term in (|14p with 174 > guarantees the "or- 
thogonality" between 0"'' and (/ij^ in the ordered phase. 
Besides the apparent Sp(4) symmetry, this free energy 
Eq. 1141 within the forth order has an extra 0(2) symme- 
try for rotation between and (^2^, which corresponds 
to the translation symmetry of the system: 



6f + i<j)f 



)exp(i27r/3). (15) 



For the commensurate V3 x -s/S order, this 0(2) symme- 
try will be broken by the sixth order terms of this free 
energy; if the noncollinear state is incommensurate, the 
0(2) symmetry will be preserved by any higher order of 
the GL theory. 

In the GL theory Eq. [TU depending on Ar — r2 ~ ri, 
the order of cj)"' is allowed to occur before the order of 
and the transition of 0° again belongs to the 0(5) 
universality class. After the order of the quadratic 
part of the free energy (|14p can be diagonalized, and 
0(3) vectors Ta,i and Ta,2 would order after 0". The 
last term in would induce a term (Taa ■ TA.2)^ at 
this transition, therefore the field theory for the second 
transition is described by the following coupled 0(3) free 
energy 



F 



\V^ri,f + r{n,f+v{{n,f + {n2YY 



+ u{{ni ■ n2Y - {niY{n2Y} + 



(16) 



with fii = Taa- Again the Goldstone mode of 0° only in- 
duces irrelevant perturbation. This coupled 0(3) model 
defined in Ref. [281] with symmetry 0(2) x 0(3) has at- 
tracted enormous analytical and numerical work, recent 
results suggest the existence of a new universality class 
of the coupled 0(3) model [i^l. When ni and n2 are 
ordered, the whole SO (3) symmetry associated with Ta^i 
is broken, and the residual symmetry of the condensate 
of fii is SO (3), which is the SO (3) symmetry associated 
with TB,i i.e. SU(2)g. 

Again the nature of the GSM depends on which type of 
SO (3) the residual symmetry is. For half-filled spin-3/2 
cold atoms on the triangular lattice, one can engineer a 
state without order of F"'', but with VS x \/3 order of 
nematic order parameter F°: 



{Vir)) - n?cos(g-r)-|-m°sin(Q-r), 



= 0. 



(17) 



This spiral nematic order parameter has residual sym- 
metry S0(3), however this is the SU(2)y subgroup dis- 
cussed previously. For instance if ni = (1,0,0,0,0) and 
n2 = (0,1,0,0,0) then SU(2)^ is generated by F34, F45 
and F35. Therefore the GSM of this order can be writ- 
ten as quotient space SO(5)/SO(3), but not equivalent to 
S"^ /Z2. The GL theory describing this nematic -s/S x \/3 
order is a coupled 0(5) sigma model, which is analogous 
to Eq. [Tg 

Another state worth mentioning briefly is the super- 
conductor state of the Sp(4) fermions, and we will only 
focus on the s— wave pairing here. The s— wave pairing of 
two Sp(4) particles can be either Sp(4) singlet or quin- 
tet. And the quintet state which is characterized by a 
complex 0(5) vector c? = di -I- 16,2 can have two types of 
GSM, depending on the microscopic parameters of the 
system. The first type of pairing has c?i parallel with c?2, 
then the GSM is [S^ x S'i]/Z2 The second type of 
pairing has di • ^2 = 0, then the GSM is again charac- 
terized by two real orthogonal 0(5) vectors, and hence 
GSM= SO(5)/SO(3), equivalent to the nematic a/Sx 
state discussed in the previous paragraph. In experimen- 
tal system with spin-3/2 cold atoms, the direct calcu- 
lation with s— wave scattering suggests that the former 
state (dubbed polar state) is hkely favored [l3|. 



3, CLOSE TO QUANTUM PHASE TRANSITIONS 

In this section we will study the phase transitions ob- 
tained in the previous section in the region close to a 
quantum phase transition. For two dimensional square 
lattice, it was proposed in Ref. 21| that by tuning J2/J1 
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in Eq. [U there is a deconfined quantum phase transition 
between Neel order and a gapped plaquette order which 
belongs to the 3d CP (3) universahty class. If now we turn 
on a weak spin interaction between square lattice layers, 
the deconfined quantum phase transition is expected to 
expand into a stable spin liquid phase with gapless pho- 
ton excitation, while the Neel order and plaquette order 
are unaltered by the weak z direction tunnelling. 

The quantum phase transition between Neel and pho- 
ton phase is described by the 3-|-ld CP(3) model: 



16e2 



9{\Za\ 



(18) 



Based on naive power counting this 3+ Id transition is 
a mean field theory with marginally relevant/irrelevant 
perturbations. To determine the universality class of this 
transition, we need to calculate the RG equation for g and 
in Eq. [TH] in detail. At the transition with r = 0, the 
coupled RG equation up to one loop for g and reads: 



dg 
dXnl 

din/ 



87r2 



47r2 



67r2 



(19) 



The RG equation for the Higgs model with = 1 was 
calculated in Ref. 3l|, the structure of the RG equation 
obtained therein is quite similar to Eq. [111 Taking this 
RG equation, one can see that the electric charge is 
always renormalized small. If one starts with a positive 
value of g, g will be first renormalized to smaller values 
marginally, and then switch sign due to its coupling with 
e^, and finally becomes nonperturbative, and no fixed 
point is found with arbitrary choices of initial values of 
g and e^. So eventually this transition is probably weak 
first order. The solution of RG equations Eq. [19] is plot- 
ted in Fig. [3] for the trial initial value go = = 1/5. 
One can see that g becomes nonperturbative much slower 
than an ordinary marginally relevant operator, because 
the ordinary marginally relevant operator will still mono- 
tonically increase under RG flow. In our current case g 



" 3d Ising 



■3d 0(8) 



Spiral Order \ Z2 Liquid 



-J2/J1 



FIG. 4: The phase diagram close to the quantum phase tran- 
sitions in stacked square (a) and triangular lattices (fe). The 
region between the dashed lines in Fig. a is the "monopole 
dominated" region which should be described by GL theory 
Eq. [T] The blue curve in Fig .b is a 3d 0(8) transition, and 
the green line is a 3d Ising transition which separates a low 
temperature classical Z2 spin liquid from a high temperature 
featureless disordered phase. 



remains perturbative and decreases for a very large en- 
ergy scale, so for sufficiently small initial values of g and 
e^, at physically relevant energy scale, we can treat this 
transition a mean field transition of spinon Zq. 

Without monopoles, the finite temperature transition 
will be described by the 3d CP (3) model in Eq. O If 
temperature is turned on, finite density of monopoles 
will be generated. Close to the quantum transition, 
since the critical temperature of the Neel order is very 
low, the monopoles roughly have small fugacity ym ~ 
exp(— i?g/r), and Eg represents the short distance en- 
ergy gap of monopole. Therefore very close to the quan- 
tum phase transition with small Tc, there is a very nar- 
row "monopole dominated" region around the classical 
phase transition where the universal physics significantly 
deviates from the CP(3) model. Inside the monopole 
dominated region the GL field theory in Eq. [7] becomes 
applicable, with r — ri+r2 tuned by temperature. Out of 
this monopole dominated region, the scaling behavior of 
the 3d CP (3) model becomes more applicable, assuming 
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the noncompact CP (3) model has a second order tran- 
sition. The size of the monopole dominated region can 
be estimated from the fugacity of the monopoles. If the 
scahng dimension of the monopole operator at the CP (3) 

fixed point is A„j, the size of the monopole dominated 

1 

range is estimated as AT/Tc ~ ym"^"'" , v is the stan- 
dard exponent of 3d CP (3) transition defined as ^ ^ r^^ . 
The phase diagram is shown in Fig. H) 

The situation is quite different for the stacked trian- 
gular lattice. In Rcf. 23] we showed that on 2d trian- 
gular lattice, by tuning J2/J1 there is a 3d 0(8) transi- 
tion between the x \fZ order and the Z2 spin liquid 
state, despite the fact that the microscopic system only 
has Sp(4)^SO(5)cSO(8) symmetry. For a stacked tri- 
angular lattice with weak interlayer coupling, both the 
■\/3 X -v/S order and the Z2 spin liquid will survive, but 
the quantum phase transition is described by the mean 
field theory of z^, because the Z2 spin liquid does not 
introduce any critical correlation for z^- Notice that at 
this mean field transition the magnetic order parameters 
0"'' will have anomalous dimension 1, because it is a bi- 
linear of Zq. For 3d space, the spin liquid can survive 
and extend into a finite region in the phase diagram at 
finite temperature, therefore close to the quantum transi- 
tion, after the thermal fluctuation destroys the magnetic 
order, the system does not enter the high temperature 
featureless phase immediately, instead it enters the finite 
temperature Z2 spin liquid phase, and the classical tran- 
sition of the spin order will simply belong to the 3d 0(8) 
universality class. At even higher temperature, there is 
a phase transition separating the classical Z2 spin liq- 
uid state and high temperature disordered phase, which 
physically corresponds to the proliferation of the "vison 
loop" . This transition belongs to the 3d Ising universality 
class. 



4, SUMMARY AND OUTLOOK 

In this work we used the Ginzburg-Landau field the- 
ory to describe and classify Sp(4) spin orders with dif- 
ferent ground state manifolds, and studied the nature of 
classical phase transitions between these spin order and 
disordered phases. Our results can be apphed to Sp(4) 
spin models like the J\ — J2 Heisenberg model in Eq. [TJ 
The GL theory can be generalized for large N spin sys- 
tems with GSM CP(N — 1), for instance the cubic term 
in Eq. [7] is always allowed by spin symmetry for large N, 
although other discrete symmetries have to be checked 
carefully. 

The monopole of the gauge field will create and 
annihilate the quantized flux of , which equals to the 
soliton number of the GSM CP(N — 1), and the existence 
of soHton of system with GSM CP(N - 1) is due to the 
fact that 7r2[CP(N- 1)] = Z for general N [3]. If we 



start with a nonlinear sigma model for CP(N — 1) mani- 
fold at 2 + £ dimension, though the spin wave excitations 
can be quite nicely described, the expansion of e at the 
phase transition will not take into account of the effect of 
monopoles. Therefore the 2 + e expansion with extrap- 
olation e ^ 1 is probably equivalent to the CP(N— 1) 
model in Eq. [3l However, if we start with a linear sigma 
model at 4 — e dimension, the 4 — e expansion will con- 
tain the information of monopoles, and the limit e — > 1 is 
likely converging to the true situation at three dimension. 
Since phase transition is what we are most interested in, 
in this work we were focusing on the linear sigma model 
in 4 — e dimension. 

Another manifold which potentially can be realized by 
Sp(4) spin system is the "squashed S"^" . The squashed 
S"^ has been studied for over two decades in high en- 
ergy theory, as one of the solutions of the 11 dimen- 
sional supergravity held equation is AdS4 x ^'J^^^^gj^ 
[3^ . The squashed S'^ is a seven dimensional mani- 
fold with the same topology as S**", but different met- 
ric and isometry group. The ordinary S"^ has isometry 
group SO (8), and the squashed S"^ has isometry group 
S0(5) X S0(3) C S0(8), and the S0(5) and S0(3) com- 
mute with each other. Written as a quotient space, the 
squashed S"^ can be expressed as [s^l 

^Jquash = [S0(5) X SO(3)J/[SO(3)^ X S0(3)^], (20) 

Here S0(3)^ is a normal subgroup of one of the S0(4) 
subgroup of the S0(5) group in the numerator, and the 
other normal S0(3) subgroup of this S0(4) is denoted 
as S0(3)b, i.e. S0(3)^ x S0{'3)g - S0(4). S0(3)^ is 
the diagonal subgroup of S0(3)^ x S0(3)p, i.e. Jf = 
jB jc^i^ 1,2,3. To realize the squashed S'^ GSM, 
we should start with a system with global symmetry 
S0(5) x S0(3). For instance, by tuning the two s— wave 
scattering lengths, the half-filled Hubbard model of the 
Sp(4) fermions can have an extra SU(2) symmetry be- 
sides the apparent Sp(4) flavor symmetry 14|. Also a 



Sp(4) spin liquid theory with fermionic spinous with mo- 
mentum space valley degeneracy can have an extra SU(2) 
symmetry contributed by the valley degeneracy. So both 
cases might be a good starting point for realizing the 
squashed S"^ manifold. We will leave the discussion of 
squashed S"^ to future study. 

The author is supported by the Society of Fellows and 
Milton Funds of Harvard University. 
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